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(54) System and method for wireiass capture of encoded data transmitted with a television, video 
or audio signal and subsequent initiation of a transaction using such data 

(57) A device (110) which can be portable and there- 
fore used in conjunction with several different data 
sources is arranged for wireless capture of encoded 
non -perceptible data directly from a light emitting de- 
vice, such as a television (101) or other video and/or 
audio output device. The data is part of the ordinary pro- 
gram displayed by the device (101) and yet indiscernible 
by a viewer or listener. The indiscernible data may be 
information needed to order those services or products, 
including prices, delivery intervals, shipping details, 
coupon offers, ancTso-on. After capture, the device in- 
cludes a memory arranged to store the data for later use, 
and communications circuitry (114 or 118) arranged to 
output at least portions of the data (or data derived from 
the captured data) to a remote agent or device 
(120,121), such as via a telephone call (via 140 or 109 
or 112) or a data transfer to a point of sale device. 




LU 



Printed by Jouve, 75001 PARIS (FR) 



BNSDOCID: <EP 0713335A2J_> 



1 



EP 0 713 335 A2 



2 



Description 

Field of the Invention 

This invention relates to a device and a method for 
the wireless capture of encoded non-perceptibie data 
from a light emitting device such as a television or other 
video and/or audio output device, and subsequent use 
of the captured data to initiate a transaction, such as 
placing a telephone call and transmitting a message to 
a remote agent or automated attendant, or delivering the 
data to a point of saie agent directly or indirectly via a 
telecommunications connection, or to operate another 
device. 

Background of the Invention - . 

In a copending application entitled "System and 
Method of Capturing Encoded Data Transmitted Over a 
Communications Network in a Video System" filed on 
behalf of Iseriberg and Tuomenoksaon March 1 S, 1 994, 
Serial No. 08/210802 and assigned to the same assign- 
ee as the present application, it was recognized that 
non -perceptible information can be encoded in a televi- 
sion program, captured in a "set top box*, and thereafter, 
used to make telephone calls. The set top box is an elec- 
trical device that is "hard-wired " directly in the circuit be- 
tween the video services network and the television set 
(as a stand alone unit or alternatively as part of the tel- 
evision set itself) and thus, unfortunately, is not portable. 
Also, the set top box is dedicated to one particular tele- 
vision (or other video source) and thus cannot be shared 
among several sources. Finally, the user of the set top 
box may be uncertain as to the nature of the information 
captured therein 

Summary of the Invention 

In accordance with our invention, a device which 
can be portable and therefore used in conjunction with 
Several different data sources is arranged for wireless 
capture of encoded non-perceptible data directly from a 
light emitting device, such as a television or other video 
and/or audio output device: The data is part of the ordi- 
nary program displayed by the device and yet indiscern- 
ible by a viewer or listener. As an example, a television 
: monitor may be located in a kiosk or other public area, 
and display a program advertising selected services and 
products. The indiscernible data may be information 
needed to order those services or products, including 
prices, delivery intervals, shipping details, coupon of- 
fers, and so-on. 

Capture can be effected by decoding subliminaNy 
modulated video broadcast signals; in a manner sug- 
gested in U. S.' Patent 4,607,031, issued on February 
21, 1989. to R S. BroughtonV et al., the teachings of 
which are incorporated herein by reference. Alternative- 
ly, a modulated carrier, which, for example, can be an 



amplitude shift keyed (ASK) or frequency shift keyed 
(FSK) signal having its primary frequency components 
at a frequency (e.g. 25 kHz) not usually present in a con- 
ventional television signal, can be added only to the lu- 

s minance component of a television signal, and the data 
can thereafter be captured using bandpass filtering and 
an ASK or. FSK receiver. After capture, the device in- 
cludes a memory arranged to store the data for later use, 
and communications circuitry arranged to output at least 

10 portions of the data (or data derived from the captured 
data) to a remote agent or device, such as via a tele- 
phone call or a data transfer to a point of sale. device. 
In accordance with the present invention, the en- 
. coded non-perceptible data includes associated dialing, 

15 routing, or identification information, so that other por- 
tions of the captured data can be transmitted to an ap- 
propriate final destination or device. In this context, the 
- final destination can be (a) a remote location, such as a 
platform arranged to process and consummate transac- 

20 tions, or (b) a point of sale system or other device (e.g., 
. TV, stereo, VCR Plus, smart mouse device, etc.). The 
other portions of the captured data that constitutes a 
payload to be communicated to the final destination or 
1 device, can, for example, be product or coupon informa- 

25 tion that is related to the display from which the non- 
perceptible data was captured, or programming or con- 
trol data. 

-.. - Further in accordance with the present invention, 
the capture device includes. a built-in display, such as 
30 .an LCD display, and the encoded non-perceptible data 
includes display information, so that at least portions of 
the captured data can be discerned by a user of the de- 
vice, and the user can thus be kept informed as to the 
■ operation of the device. The memory and the commu- 
35 ;nications circuitry in the capture device are advanta- 
geously arranged so-that, in a transaction initiated by 
* the device using captured information, a two-way dialog 
\ may be initiated between the user and an agent at a re- 
mote terminal or a POS system, and information may 
40. be transmitted to and stored in the device for later re- 
trieval. 

Brief Description of the Drawing 

45 ■ . The present invention will be more fully appreciated 
by consideration of the following Detailed Description, 
which should be read in light of the accompanying draw- 
ing in which:; 

Fig. 1 is a schematic drawing of , one embodiment 
of a system arranged in accordance with the 
present invention for wireless capture of encoded 
data transmitted with a television signal; 
Fig. 2 is a block diagram illustrating the components 
of capture device 110 of Fig. 1; 
Fig; 3 is a diagram illustrating the format of captured 
data; and 

Fig. 4 is a flow diagram of the process by which data 
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embedded in a subliminal visual or non-audible 
message is captured and used. 

Detailed Description 

Referring first to Fig. 1 , there is shown a schematic 
drawing of one embodiment of the present invention ar- 
ranged for the wireless capture of encoded non-percep- 
tible data from a light emitting device such as television 
101 , and subsequent use of the captured data to initiate 
a transaction which might be handled by a person at an 
agent terminal 1 20, or a person at another terminal 1 21 
thai is supported by an interactive service platform 1 30. 

A television signal is received by television 101 via 
over-the-air transmission, as from transmission tower 
102 via a transmission link 103, or alternatively is sup- 
plied by a cable TV connection, or from a-VCRA/ideo- 
tape. The video portion of the television signal is dis- 
played on the piciure tube or other visual display area 
105 of a monitor, computer or a television 101, while the 
audio portion of the television signal is played through 
or by projected from speaker 106. Associated with the 
program that is seen and/or heard by a viewer/listener 
is non-discernible encoded data that is transmitted as 
part of the audio and/or video signal. For example, the , 
program may be an advertisement for a service or prod- 
uct, and the indiscernible data may create a subliminal 
visual pattern on the visual display area, that when prop- 
erly received and decoded, contains information need- 
ed to order those services or products, including prices, 
delivery intervals, shipping details*; coupon offers, and 
so-on. 

Encoding, transmission and decoding of non-dis- 
cernible data may be accomplished as described in a*: 
patent application entitled "System and Method for En- 
coding Digital Information in a Television Signal" filed on 
behalf of T Sizer and assigned to the same assignee as 
the present application, which cd-pending application is 
incorporated herein by reference; Briefly, it was found 
using a spectrum analyzer on r a typical video signal, that 
there are comparatively large frequency components at 
the line rate and at the frame rate and its harmonics, but, 
that between these frequencies, there are other fre- : . 
quency bands in which little information is carried. One^ 
such frequency band is between 1 5 and 30 kHz. By add- 
ing a low level carrier signal or tone at a frequency in 
this band, say 25kHz, the video image is not degraded, 
but a properly tuned decoder can receive and decode 
the encoded information. In this way, digital information 
can be subliminally inserted in a video signal by adding 
to the video signal an amplitude shift keyed (ASK) or 
frequency shift keyed (FSK) carrier signal, and the dig- 
ital information can later be recovered using ASK or FSK 
decoding. Encoded data can also be inserted in a tele- 
vision signal and recovered by a receiver that responds 
to the picture displayed on the television, in the manner 
described in patent 4,807,031 cited above. 

The program displayed on television 101 is sensed 



by a portable capture device 110, which is described in 
more detail in connection with Fig 2. When the non-dis- 
cernible encoded data is part of the video signal, such 
sensing takes the form of tight from display area 105 

5 being collected by an optical. lens 117, while when the 
non-discernible encoded data is part of the audio signal, 
such sensing takes the form of sound from speaker 1 06 
being collected by a microphone (211 in Fig. 2). In either 
event, there is a wireless communication of the informa- 

10 tion from television 101 to capture device 1 1 0 such that 
the encoded data is "viewed" or "heard" rather than be- 
• ing transmitted through, a wired connection, or through 
a more conventional radio frequency communication. 
In addition to receiving non-discernible encoded da- 

is ta, capture device 110 has the capability of (a) storing 
the encoded data, (b) t transmitting (or otherwise output- 
ting) the encoded data "(or information retrieved using 
such data) to a remote location or device, and (c) dis- 
playing the encoded data to a user. The data may in- 

20 . elude information for initiating a transaction or originat- 
ing a telephone call, as well as additional information (e. 
g., price and ordering information) associated with the 
television program being displayed on television 101. 
The remote location may be agent terminals 1 20 or 1 21 , 
. 25 interactive service platform 130, or a point of sale sys- 
tem, and the communication between the capture de- 
i vice 110 and the remote, location may be. one-way or 
two-way. 

I n the case where encoded data is to be output from 

30 capture device 110 via a telephone call, the call may be 
initiated and transacted in one of several ways, as illus- 
trated in Fig. 1 . First, the data may be transmitted over 
a wireless link to an associated telephone base station 
-., 109 that is, in turn, connected to a, telephone line 111 

35 that is part of a telecommunications network 115. Com- 
munication between capture device 110 and base sta- 
tion 109 may be accomplished in a variety of ways; as 
an example, the same signaling and transmission can 
be used as is presently used in communication between 

. 40 a cordless telephone handset and its associated base 
station. . Second, capture device 110 may be directly 
connected to a telephone line 112 through an interface 
that includes a standard telephone jack 1 1 4. Third, cap- 
- ture device .110 may include the functionality of a cellular 

45 telephone, so that the data may be communicated to a 
remote location via a wireless call placed to a mobile 
.telephone switching office (MTSO) 140, which is in turn 
connected to telecommunications network 115. 

As explained more fully in connection with Fig. 2, 

50 capture device 110 may include a dual tone multi-fre- 
quency (DTMF) transceiver arranged to generate a se- 
ries of audible tones. Accordingly, a fourth way to initiate 
. a telephone call would apply portions of the data to the 
transmitter (tone generator) portion of the DTMF trans- 
ss ceiver. Capture device 1 10 may then be placed in juxta- 
position with the microphone portion of a conventional 
telephone handset, such that the audible tones gener- 
ated in capture device 110 are acoustically coupled 
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through the handset to receivers in telecommunications 
network 115. 

Capture device 110 can also output encoded data 
directly, without making a telephone call. This is useful 
in conjunction with a point of sale system or the like, 
which includes a docking station or other interface to 
which the capture device may be connected, or an IR 
receiving device or a device responding to DTMF. 

If an outgoing telephone call is initiated by capture 
device 110, the call can illustratively be routed through 
telecommunications network 115 directly to an agent 
terminal i 20, or to an interactive service platform 1 30 to 
which is connected another agent terminal 121. When 
a connection is completed to platform 130, the call may 
be routed by a switch matrix 1 31 to a voice response 
unit 1 32 that operates under the control of a processor 
1 34. Information included in the captured encoded data 
can be provided to the attendants, or processor 134 can 
perform a look-up and retrieval operation in database 
1 31 , using encoded data as a key, in order to effectuate 
a desired transaction. 

For certain transactions, information connected 
with a transaction may also be transmitted from service 
platform 1 30 to capture device 110, either for display to 
a user, storage for later use; or output to a point of sale 
system or device. This information may for example, be 
confirmation information, indicating that a transaction 
has been effected, or details regarding a just completed 
transaction, such as ticket information relating to a travel 
reservation. 

Referring now to Fig. 2; there is shown a block dia- 
gram illustrating in more detail the components in one 
embodiment of capture device 110 of Fig. 1 If the en- 
coded rion -perceptible data is inserted in the video por- 
tion of a television signal, the images displayed on visual 
display area 105 are captured by optical lens 117 and 
an associated photodetector 203] which is arranged to 
supply an electrical signal to ; filter/receiver 215 repre- 
senting the image. On the other hand, if the encoded 
noh -perceptible data is inserted in the audio portion of 
" a television signal, the output from speaker 106 is cap- 
, tured by a microphone 211 and an associated amplifier 
213, which is arranged to supply an electrical signal to 
filter/receiver 215 representing the sound energy. In ei- 
ther event, the encoded non-perceptible data is decod- 
ed in filter/receiver 2i 5 in a manner consistent with the 
manner in which the original data was encoded. Thus, 
filter/receiver 215 can be arranged to perform the same : 
functionality as the elements illustrated in Fig 4 of the 
above referenced Broughtori et al. patent; or, alterna- 
tively, the operation of filter/receiver 215 can be as de- 
scribed in conjunction with Figs. 4 or 7 in copending ap- 
plication of T.Sizer. in each instance, it is noted that cap- 
ture device 1 1 0 receives and recovers encoded data by 
"watching" or "listening to" the output of television 101 
(or another similar display device), and that the capture 
device 110 is not connected to television 101 or the other 
display device by a cable or wires. 



< The output of filter/receiver 215, representing the 
data captured by capture device 110, is applied to a mi- 
croprocessor 220, which is arranged to perform various 
data processing and control functions in coordination 

5 with programs stored in an associated memory 221 as 
well as inputs received from a user of capture device 
1 1 0 Generally speaking, capture device 1 1 0 may oper- 
ate in an active or a passive mode. In the active mode, 
capture device 110 responds only to user initiated cap- 

w ture commands. In the passive mode, capture device 
110 captures all of the data sent to it while in that mode. 
■ In either mode, captured data may be routed by micro- 
processor to memory 221 and stored for use at a later 
time, as determined by the user. The captured data may 

75 also be displayed on a display 225, such as a liquid crys- 
' tal display, so that a user wilt be aware of exactly what 

• * information was captured. 

The control programs contained in memory 221 can 
implement numerous (unctions, which, broadly speak- 

zo ing, can be categorized as (a) functions relating to data 
*" capture, such as SAVE and AUTOSAVE functions, (b) 
functions relating to output of already captured data, 
such as DIAL and SEND DATA functions, (c) functions 
relating to memory manipulation, such as the RECALL 

'2S and DELETE functions. The above enumerated func- 
tions are initiated when respective buttons 241 - 246 are 
activated, and the activation signal output from a button 
is applied to microprocessor 220 via a button interface 
240. Each of the functions is discussed below. 

so - When the SAVE function is initiated by activation of 
button 241, capture device 110 is in the active mode, 
and the encoded data contained in the current audio or 
video output of television 101 is captured. This is ac- 
complished, for example, by activating optical lens 117 

35 . and photodetector 203, or microphone 211 and ampli- 
fier 21 3; for a relatively brief period of time, typically on 
' ■ » ■ the order of several seconds, so that encoded data as- 
sociated with the current program being displayed on 
- display area 105 or played through the loudspeaker 1 06 

40 is processed byfilter/receiver215and stored in memory 
221 . A visual or auditory cue may be contained in the 
video or audio program, to indicate to a user of a capture 
device that there is data to be captured. When the SAVE 
function is activated, microprocessor 220 may be ar- 

45 ranged to control display 225 so as to indicate the cur- 
rent status, e.g., that the capture device is in the process 
of receiving and capturing data. When the data has been 
captured, microprocessor 220 may be arranged to con- 
trol display 225 so as to actually display some or all of 

50 the data, so that the user is now aware of information 
that had theretofor been subliminal and indiscernible. 
" The display can also indicate that the device is "aimed" 
correct ly. 

When the AUTOSAVE function is initiated by acti- 
os vation of button 244, capture device 110 is in the passive 
mode. In this mode, capture device 110 captures a se- 
ries of encoded data that may be contained in each of 
a series of programs that can be viewed/heard on tele- 
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vision 101. This is accomplished, for example, by acti- 
vating optical lens 117 and photo detector 203, or mi- 
crophone 21 1 and amplifier 213, for a relatively long pe- 
riod of time, typically on the order of several minutes or 
hours. Encoded data captured from the various pro- 
grams being displayed on display area 105 or played 
through loudspeaker 1 06 over a long period of time are 
processed by filter/receiver 215 and stored in memory 
221 . As with the SAVE function, when the AUTOSAVE 
function is activated, microprocessor 220 may be ar- 
ranged to control display 225 so as to indicate the cur- 
rent status, e.g., that the capture device is in the process 
of receiving and capturing data. 

When a user of capture device 110 desires to use 
any of the data stored in memory 221, including data - 
stored a result of the AUTOSAVE function, the contents 
of memory 221 may be retrieved and displayed on dis- 
play 225 by initiating the RECALL function upon activat- 
ing button 245. This permits a user to be aware of ex- 
actly what information was previously captured and 
stored. Through this function, the user will -be able to 
selectively peruse or scroll through the stored data, so 
as to retain or use selected portions thereof. Activation 
of button 246 initiates the DELETE function, whereby 
portions of the data stored in memory 221 are selectively . 
erased. In order to conserve. and thereby "recycle" to , 
storage capacity of memory 221, microprocessor 220 
may also be arranged to manage the contents in mem- 
ory 221, so that the oldest information . stored therein, - 
could be cycled out and deleted as new information is 
captured and stored. 

Turning now to consideration of the use of already 
captured data, it is an important functionality of .capture 
device-1 1 0 to permit such data to be output upon a com- 
mand from a user. Such data, when output,. can be used r 
to (a) make a telephone call, and, when the call is an- 
swered, to provide information to : a ; live or automated 
attendant that receives the call, or (b) initiate a transac- . : 
tion, such as by interacting with a special purpose ter- 
minal or kiosk arranged to receive the stored data,. or a . 
device such as a VCR Plus or a stereo: In accordance 
with the arrangement shown in Fig. 2, data output via a. 
telephone call can be initiated in several ways. 

First, captured data can be routed by microproces- . 
sor 220 from memory 221 to a modem 246 and then to . 
a base station interface 250, so that the information can 
be communicated to telephone base ,1.09 of Fig. 1. This 
in turn allows a conventional telephone call to be origi- . 
nated from telephone base 109. . 

Second, captured data can be routed through mo- 
dem 246 to a wire interface 253, which allows capture 
device 11 0 to be physically connected to a terminal or 
docking station for information off loading. 

Third, captured data can be routed to a DTMF trans r 
ceiver 247, which includes a tone generator arranged to 
output DTMF tones. These tones can be directly con r 
nected to a telephone line 1 1 2 via a standard connection 
such as RT1 1 jack 1 1 4. Alternatively, capture device 1 1 0 



can be placed in juxtaposition with the microphone in a 
telephone handset, and the audible tones can be ap- 
plied and used much like an audible frequency autodi- 
aler. 

5 Fourth, captured data can be routed through mo- 
dem 246 to a cellular telephone 252, which can initiate 
a cellular call. When a cellular connection is completed, 
the captured data can be output in encoded form or as 
audible tones. 

10 For each of the arrangements just described, a tel- 
ephone call may be originated using the DIAL function. 
This function is initiated by activation of button 242. With 
this function, dialing or routing information, which is part 
of the data captured by capture device 110 and now 
is stored in memory 221, As retrieved. This information is 
then further processed, so that a telephone call can be 
initiated. In some instances, a call is originated by ap- 
plication of stored data.to DTMF transceiver 247, so that 
. audible tones can be output. In other instances, a data 
2Q call can be initiated by appropriately formatting stored 
data. In yet other instances, such as when a cellular call 
is dialed, stored dialing information may be applied to 
an, outgoing number register. The exact manner in which 
information is output from capture device 110 can vary, 
25 in accordance with several embodiments ofjhe present 

invention 

While the DIAL function is used to output certain 
captured information needed to i nitiate a telephone call 
. (or other data output function), the SEND DATA function, 
30 initiated by activation of button 243, is used to output 
other portions of the captured data stored in memory 
221, pertaining, for example, to a transaction that the 
user desires to make. This additional data is retrieved 
from memory 221 and appropriately transmitted, de- 
■35 pending upon the manner and type of telephone call in- 
volved. Thus, data may be output either via base station 
interface 250 to telephone base 109, or via RJ 11 jack 
:11 4 onto telephone line 11 2, or via cellular phone 252 
. - to MTSO 140. Note that if output occurs directly, without 
40 a telephone call, the SEND DATA functibn may cause 
information from memory 221 to be output via wire in- 
terface 253. During the SEND DATA function, the infor- 
mation being output may also be displayed on display 
225. This allows a person .using the capture device to 
45 identify the information that was captured, and use the 
information in an actual transaction. 

it is to be noted here that capture device 1 10 of Fig. 
2 may optionally include an automatic speech recogni- 
tion circuit 245, an infrared. (I R) transmitter 235, a hand- 
so set 230 and a keypad 247. The purpose of I R transmitter 
235 is to allow captured data.to be output directly to a 
. device such as a point of sate system or a VCR or the 
, like, via infrared transmission. The purpose of speech 
recognition circuit 245 is to convert captured data to spo- 
55 ken phrases, which can be played to a user or to an at- 
tendant after a telephone call is initiated. The purpose 
of handset 230 and keypad 247 is to allow capture de- 
vice 110, in addition to being used in accordance with 
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the present invention, to be used to complete conven- 
tional dialed telephone calls. 

In one application of the present invention, a tele- 
vised advertisement for a product includes additional or- 
dering or "coupon" information embedded. in a sublimi- 
nal visual pattern within the video image displayed on 
visual display area 105. The coupon information de- 
scribes the item(s) offered for sale, the regular price, the 
coupon value, and contains additional information need- 
ed for ordering. This embedded information is transmit- 
ted between television 101 and capture device 11 0 at a 
rate such as 60 bits per second; that is sufficient such 
that the necessary information can be transmitted in a 
very brief period of time. In the application just de- 
scribed, the embedded product information, dialing 
number, coupon information and necessary checkbits 
together comprise approximately 2,000 bits of informa- 
tion, so that between three and four seconds would be 
required for transmission. Once the coupon information 
is stored in capture device 110, the user of the capture ; 
device can supplement the coupon information with 
user entered data which can be entered using keypad 
-247. This supplemental information may include credit : 
card or other billing information pertaining to the user, 
as an example. , . . : 

As shown in Fig. 3, the information represented by 
the encoded data can be organized in severalfields. Us- 
ing the coupon application as an example, a first field 
301 contains a telephone number which may be used; 
to originate a telephone call in order to order the product 
to which the coupon applies. Illustratively,. this will be a 
toll-free or "800" number. Other information associated 
with the coupon can include a product description in field 
303, an expiration date in field 305, the time that the 
advertisement was televised in field 307; the number of 
the television channel running the ad in field 309, and a 
serial number for the ad in field 311, identifying either or . 
both the particular ad and its sequence in an ad cam- 
paign. It is to be noted that various other information can 
also be included in the information illustrated in Fig. 3, 
such as keywords'or attributes describing the received 
encoded data, as well as information indicating that a 
product is available in particular colors or sizes. The key- ; 
words would be useful in retrieving from memory 221, 
information relating to specific products or advertisers. 

The process by which information embedded in a 
subliminal visual or audible message is captured and 
used is illustrated in flow diagram form in Fig. 4. As in- 
dicated previously, two general modes of operation are 
contemplated, namely a "passive" mode and an "active- 
mode, both of which follow a "start" step 401 . 

In the passive mode, a user desires to operate cap- 
ture device 1 10 in an unattended manner, andtocapture - : 
all of the encoded data that may be subliminally dis- 
played on a display device 105 or output from speaker 
1 06 over a relatively longperiod of time, typically several 
minutes or hours, until the user, at some later time, de- 
sires to discontinue or terminate the capture This mode 



is initiated using AUTOSAVE functionality, by activating 
button 244. When this is done, a positive response is 
obtained in step 502, so that data capture, storage and 
: display, all as described below in connection with step 
s 507, occur. While in, the passive mode, capture is termi- 
nated by subsequent activation of the same button 224. 

lathe active mode, a user operates capture device 
110 in real time,. typically while viewing a video display 
or listening to an audio presentation. During the display 
10 or presentation, the user sees or hears a corresponding 
televised advertisement or other indication that imper- 
ceptible information is available for capture. The user 
may, at this time, be instructed. to activate SAVE button 
241, or the user may know, from previous experience, 
is how to use the device. When button 241 is activated, a 
positive result occurs in step 503, thereby capturing the 
; imperceptible encoded data, displaying all or a portion 
of the data on display 225 ; and storing.same in memory 
, ; 221, all.in step 507. Until the SAVE button is activated, 
20 capture device 110 may be in a "wait state" 505. 

The .user of capture device 1 1 0 determines that the 
appropriate data has been captured by reviewing the in - 
. formation seen in display 225. After the captured data 
has been reviewed and determined to be correct, acti- 
os vation of the DIAL button 242 on the portable capture 
device 1 1 0 in step 509 initiates step 51 3, by which stored 
dialing and/or routing information, illustratively in field 
401 of Fig. 4, is recalled from memory 221 . A call is then 
. originated launched, such by transmitting the dialing in- 
30 . formation from memory 221 through base station inter- 
face 250 to base station 109 and thence to telephone 
line 111 and telecommunications network 115 to inter- 
.. active service platform 130. The call may be a local or 
long distance call, made using a. "plain old telephone 
35, service" (POTS) line; alternatively, a toll-free call may 
. belaunched using an 800 number. Desirably, automatic 
. number identification (ANI), caller I'D, and other features 
available in the telecommunications network may be 
used to proyide ; additional information to the called party, 
40 in addition to the captured data that is transmitted in suc- 
, ceeding steps in the process. Until the DIAL button is 
activated, capture device 110 may be in a "wait state" 

.;. -511. 

After the call has been originated, a determination 
45 is made in step 51 5 that the call has been answered and 
' that the SEND DATA button 243 has been pushed. (Until 
■ the SEND DATA button is activated, capture device 110 
may be in a "wait state" 517.) When a positive result 
occurs in. step 515, information contained in fields 
50 403-411 is transmitted to the called destination in step 
519. As. stated previously from the point of view of the 
cailed party, the information received includes details 
- ; . • concerning the advertised product, the identity of the 
caller, and other information associated with the item 
£5 that was initially displayed or mentioned on the televi- 
* ston program Advantageously the advertiser will know 
more about their customer than ever before. This is be- 
cause the advertiser will be able to determine from the 
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transmitted information which advertisement drew the 
business, how long it took the caller to respond to the 
ad. and the geographic location of the callers, based 
upon ANI or caller ID. It will be easy to tell which ads in 
a campaign are most successful. 

In accordance with one optional arrangement of the 
present invention, when data" is captured and stored in 
step 507, the newiy collected data is compared with pre- 
viously stored data. When a match is found, any dupli- 
cate data is dropped. In addition, the information in var- 
ious fields is examined, to assure that the data captured 
is in the appropriate format. If an error is detected in any 
data, the information may be deleted, and "new" data 
collected. Tnis is accomplished by repeating capture 
step 507 several times. 

In another application of the present invention, an 
interactive television capability is provided in connection 
with a game show or television drama. For a game show, 
an audience would be asked to vote on various aspects 
of the game. This could include questions such as which -* 
person should goon a date with the contestant, or which 
singer is the best? The audience would be instructed to 
press the "SAVE" button on their capture device at the 
appropriate time specified for their choice/ The viewers 
vote would then be cast by depressing the DIAL button, 
communicating with the game show host, and then 
transmitting information indicating the vote by depress- 
ing the SEND DATA button. Alternately, one number 
could be transmitted and an automated interactive voice 
platform such as the InfoWorx™ platform available from 
AT&T could offer options to the caller. These calls could 
be toll free calls, or, alternatively, billed to the caller. 

In yet another application of the present invention, 
a video on demand capability can be provided. A viewer 
sees a video he or she would like to see advertised on 
a previewing network. An instruction to press the DIAL 
button on the capture device is made on the previewing 
channel to indicate to the viewer that he or she can press 
the button to order a particular video. The call would be 
placed to a voice response system' used to reserve the 
video on demand: Information about which video the 
caller wanted to see is sent over the phone lines once 
the call is connected. The caller would merely have to 
acknowledge the order and select a time slot. 

The present invention can also be employed in the 
context of programming of a VCR, particularly where the 
user already has the capabilities provided by VCR Plus, 
a product that is commercially available. In this embod- 
iment, a code sequence associated with and represent- 
ing a particular television show constitutes the informa- 
tion included in a televised advertisement. When the ad 
is viewed, the viewer can press the SAVE button and 
send the sequences to the VCR Plus using 1R transmit- 
ter 235. 

While trie portability of capture device 11 0 is advan- 
tageous, it is to be noted that in some arrangements, 
the functions of capture device 110 and telephone base 
109 may be combined in a single stationary unit, which 
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might resemble a cordless telephone base station. In 
this event, the functionality of capture device 110 would 
be included in the stationary. unit, and that unit would be 
arranged, in the manner previously described, to collect 
information from the visual display area 105 and to pro- 
vide storage of the collected information. In this arrange- 
ment, the stationary unit could also be adapted to utilize 
input devices such as optical character recognition 
(OCR) or scanners that would provide an additional 
method of inputting teiephone numbers and coupon 
and/or product information. The hand piece of this sta- 
tionary unit would be used to place a call and send the 
data.- ■ '* ■ . : 

The present invention is not limited to live televised 
programs, but can also be, operated in conjunction with 
stored video programs played from a VCR. In this ar- 
rangement, advertisements and other encoded informa- 
tion is included in videos at the time they are recorded 
1 or at the time they are mastered and put onto video cas- 
settes. A viewer watching a video with such an ad can 
respond immediately to reserve another video, pur- 
chase one of the items advertised, or order catalogs au- 
tomatically. When a program containing sublirninally en- 
coded data is viewed, the viewer simply^presses the 
DIAL button, launching a telephone call, and then press- 
; es the SEND DATA button, sending captured data, such 
- as product information, to an agent, who can verify the 
order using the additional information that was captured 
or stored in the device ?r 

The portability of the present invention can be used 
to advantage in implementing what can, in effect, be 
called a "Roaming Bulletin Board*. A user can be attend- 
ing a conference or traveling through jhe airport where 
kiosks or bulletin boards are set up. In the case of a con- 
ference, an attendee would walkthrough the conference 
viewing the displays. When the attendee sees a display 
of interest, he or she points the capture device towards 
the display, and captures the non-discernible encoded 
■- data from either the displayed image or the sound out- 
put. Later, the user launches a telephone call to the tel- 
ephone number specified and included in the captured 
data r and sends out coupon information that is also part 
of the captured data. 

, It is further to be noted that the present invention 
can be used in the context of automobiles and mobile 
telephones. When a capture device in accordance with 
the invention is placed in a car, it can receive and cap- 
ture non-discernible encoded data contained in the au- 
dio output of the car radio. The captured data can in- 
clude a toll free or conventional telephone number for 
originating a mobile cellular telephone call from the car. 
. The call may be placed to an interactive service plat- 
: form, and additional captured data, such as coupon or 
similar information, can then be used to effectuate a 
transaction. 

Various modifications and adaptations of the 
present invention will be apparent to those skilled in the 
art. For example, the functions provided by the present 
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invention may be combined with those used in a per- 
sonal digital assistant (PDA) or a laptop computer. If de- 
sired, a simultaneous voice and data (SVD) modem of 
the type available now from AT&T could be provided in 
capture device 110 to allow communication both by 
voice and data transfer, at the same time. Various func- 
tions can also be added to the processes and applica- 
tions described. For example, telephone base 109 may 
be programmed to dial a number or series of numbers 
at specific times, when certain amounts of information - 
or data have been collected, or based upon some crite- 
ria. Capture device 110 could then send collected infor- 
mation "in bulk" to interactive service platlorm 130. For 
this purpose, modem 246 could be used, so as to send 
the information more efficiently. 



shift keyed (ASK) or frequency shift keyed (FSK) 
carrier signal added to a television signal, and 
wherein captu re is effected by detecting and decod- 
ing said carrier signal. 

5 

7. A system for capturing data comprising 

means for wireless capture of indiscernible 
data embedded in an audio/video display; and 
■io . means for outputting at least portions of said 
captured indiscernible data to a remote device 
via a telephone call. 

8. Apparatus for initiating a transaction relating to a 
"is - program appearing , on a television display, said 

apparatus comprising 



Claims ■ means arranged for wireless capture of data 

directly from said television display, said data 

1 . Apparatus for initiating a transaction, comprising 20 being indiscernible to a viewer of said television 

display; 

means arranged for wireless capture of data : * ^ means responsive to said captured data for 

directly from an audio/visual display, said data obtaining a telephone number, said telephone 

being indiscernible by a viewer or listener, and number being associated with a remote loca- 

means arranged to transfer at least portions oi 2s . tion arranged to perform transactions associ- 

said data to a separate transaction processor ated with said program appearing on said tele- 

to initiate said transaction. vision display; 

■ means for initiating a telephone call using said 

2. The invention defined in claim 1 wherein said data ~ telephone number; and 

includes dialing information for originating a tele- 30- means for transmitting to said remote location, 

phone call, and . wherein said transfer means ' ^ during said telephone call, additional informa- 
includes means for initiating said telephone call tion contained in said captured data, said addi- 

using said dialing information. tional information also pertaining to said pro- 

' ■ ■ ; gram. 

3. The invention defined in claim 1 wherein said appa- 35 - 

rat us further includes 9. A system for. responding to a televised advertise- 

ment, comprising 

means responsive to said capture means for 

storing said data, and : means for embedding non-discernible informa- 

wherein said transfer means includes means 40 tion in said advertisement, said information 

for coupling information from said storing including (a) a telephone number for a remote 

means directly to a separate transaction proc- processor, arranged to process orders for the 

essor. items displayed in said advertisement, and (b) 

other information relating to said items dis- 

4. The invention defined in claim 1 wherein said cap- 1 played in said advertisement; 

ture means is in a portable capture device and said " ■ - means arranged for wireless capture of said 

transfer means is in a base station, and wherein' non-discernible information directly from said 

said apparatus further includes means for transfer- televised advertisement; and 

ring information from said portable capture device : means responsive to said captured information 

to said base station. 50 for (a) placing a telephone call using said tele- 

• phone number and (b) transmitting said other 

5. The invention defined in claim 1 wherein said information to said remote processor via said 
audio/visual display includes non-perceptible telephone call. 

encoded data, and wherein said capture means is 

arranged to receive and decode said encoded data. 55 10. A method for initiating a transaction, comprising the 

steps of 

6. The invention defined in claim 5 wherein said non- 
perceptible encoded data includes an amplitude capturing data directly from an audio/visual d is - 
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play, said data being indiscernible by a viewer 
or listener, and 

transferring at least portions of said data to a 
separate transaction processor to initiate said 
transaction. 

11. The method defined in claim 10 wherein said data 
includes dialing information for originating a tele- 
phone call, and wherein said transferring step, 
includes initiating said telephone call using said 
doling information. 

12. a me trod for capturing data comprising 

capturing using wireless capture, indiscernible 
an:* embedded in an audio/video display; and 
ou'.putting hi least portions of said captured 
indiscernible data to a remote device via a tel- 
ephone call 

20 

13. Amuthodfor initiating a transaction relating to a pro- 
gram appearing on a television display, said method , 
comprising steps of < . 

using wireless capture data directly from said 2S 
television display said data being indiscernible 
to a viewer of said television display; 
responsive to said captured data, obtaining a 
telephone number, said telephone number-, 
being associated with a remote location ,30 
arranged to perform transactions associated 
with said program appearing on said television 
display; 

initiating a telephone call using said telephone 
number; and t 3S 
transmitting to said remote location, during said 
telephone call, additional information con- 
tained in said captured data, said additional , 
information also pertaining to said program. 

; -, ; - 40 

14. A method for responding to a televised advertise- 
ment, comprising the steps of 



THIS PAGE BUNK (uspto, 



embedding non-discernible information in said 
advertisement/said information including (a) a. 45 
telephone number for a remote processor 
arranged to process, orders for the items dis- 
played in said advertisement, and (b) other 
information relating to said, items displayed in . 
said advertisement; . 50 

capturing, using wireless capture, said non-dis- 
cernible information directly from said televised 
advertisement; and - 

responsive to said captured information, (a) . 
placing a telephone call using said telephone ss 
number and (b) transmitting said other informa- 
tion to said remote processor via said tele- 
phone call. 
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